DTE AICCOMAS 2025

Multimodal Deep Learning for Dynamic and Static Neuroimaging: Integrating MRI and fMRI for Alzheimer’s Disease Analysis

  • Kujur, Anima (IWR, Heidelberg University)
  • Monfared, Zahra (IWR, Heidelberg University)
  • Dietrich, Felix (Technical University of Munich)

Please login to view abstract download link

In the realm of neuroimaging, Magnetic Resonance Imaging (MRI) provides detailed anatomical insights, while functional MRI (fMRI) captures real-time brain activity. Traditionally, these modalities have been analyzed separately, limiting the ability to fully understand the complex interplay between brain structure and function. In this contribution, we discuss a deep learning framework that integrates multimodal MRI and fMRI data to simultaneously predict structural abnormalities and analyze functional brain activity. By combining the strengths of both modalities, our model captures static anatomical patterns from MRI alongside dynamic functional signals from fMRI, offering a comprehensive analysis of brain health. This joint learning approach enhances predictive performance in diagnosing and monitoring Alzheimer’s disease, where structural degeneration often correlates with altered brain activity. The framework involves acquiring and preprocessing both MRI and fMRI data, extracting static structural features from MRI and dynamic functional features from fMRI, and then fusing these modalities using a deep learning model capable of spatial-temporal integration. In addition, we employ generative AI methods to synthesize data, such as generating MRI images from fMRI or vice versa, enabling a richer multimodal dataset for training. Through joint learning, the model predicts both structural abnormalities and functional brain activity, optimized using shared loss functions. By leveraging deep learning architectures capable of fusing both spatial and temporal features, this framework offers a comprehensive approach to brain health assessment and disease progression monitoring for Alzheimer’s Disease. The proposed multimodal data-driven approach can pave the way for more accurate, reliable, and comprehensive neuroimaging solutions, offering deeper insights into brain function and structure simultaneously.